• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    An augmented reality content generation method for real-world equipment (E) employing a mobile terminal (1) having a screen (11), the method comprising: A determining step an augmented reality database stored in an augmented reality server (S1), a step of reading with the mobile terminal a matrix code (20) or an electronic tag associated with said equipment (E ), said matrix code (20) or said electronic tag respectively encoding or comprising an address linked to a content, a step of storing said address, a step of generating a graphical interface component (Wx) corresponding to said address providing access to said content, A step of storing the GUI component (Wx) in the database (DB) stored in the augmented reality server (S1).
    公开号:FR3019333A1
    申请号:FR1452555
    申请日:2014-03-26
    公开日:2015-10-02
    发明作者:Tewfik Meftah；Benoit Jacquemin
    申请人:Schneider Electric Industries SAS；
    IPC主号:
    专利说明:

    [0001] Technical Field of the Invention The present invention relates to a method of generating augmented reality content for real world equipment.
    [0002] STATE OF THE ART It is known to affix matrix codes or electronic tags on equipment. With the aid of a mobile terminal, an operator can scan this matrix code which is then decoded or read the electronic tag. The decoded or stored address then returns to a content associated with the equipment. This content is stored on a remote server to which the handheld connects to access it. Access to content related to the matrix code or electronic tag is not always easy. The main reason is that the matrix code or the electronic tag can only be read by the mobile terminal at a very short distance, requiring the equipment to be accessible. This is not always the case, for example when the equipment is housed in a cabinet. The object of the invention is to propose a method allowing a user to access content related to the matrix code or the electronic tag, even if it is not easily accessible. SUMMARY OF THE INVENTION This object is achieved by an augmented reality content generation method for a device located in the real world, by using a mobile terminal equipped with a screen, the method comprising: A step of determining a augmented reality database stored in an augmented reality server, a step of reading with the mobile terminal a matrix code or an electronic tag associated with said equipment, said matrix code or said electronic tag respectively coding or comprising an address linked to a content, a step of storing said address, a step of generating a graphical interface component corresponding to said address for accessing said content, a step of storing the graphical interface component in the database stored in the augmented reality server.
    [0003] According to a particularity, the matrix code is a QR code or equivalent. According to another particularity, the electronic tag is of the NFC or RFID type. According to another feature, the method comprises a step of associating the graphical interface component with the equipment or with a marker affixed in the vicinity of the equipment. In another feature, the step of associating the graphical interface component with the equipment is performed by moving the graphical interface component displayed in augmented reality on the screen of the mobile terminal on an image of the displayed equipment. on the handheld screen or in a geographical landmark. In another feature, the GUI component is composed of a text or a symbol. The invention also relates to an augmented reality content generation system for equipment located in the real world, this equipment comprising: a module for determining an augmented reality database stored in an augmented reality server; A module for reading a matrix code or an electronic label associated with said equipment, said matrix code or said electronic tag respectively encoding or comprising an address linked to a content, said electronic tag being associated with a content, a memory module of said address, a module for generating a graphical interface component corresponding to said address for accessing said content, a storage module for the graphical interface component in the database stored in the augmented reality server. According to a particularity, the matrix code is a QR code or equivalent. In another feature, the system includes an association module of the graphical interface component with the equipment.
    [0004] According to another particularity, the association module of the graphical interface component with the equipment is arranged to allow a sliding of the graphical interface component displayed in augmented reality on the screen of the mobile terminal on an image of the equipment. displayed on the handheld screen. In another feature, the GUI component is composed of a text or a symbol.
    [0005] BRIEF DESCRIPTION OF THE FIGURES Other features and advantages will appear in the following detailed description with reference to the accompanying drawings in which: FIG. 1 represents a system used to implement the method of the invention, FIGS. 2A to 2D illustrate schematically the various steps of the method of the invention.
    [0006] DETAILED DESCRIPTION OF AT LEAST ONE EMBODIMENT The invention relates to a method for generating augmented reality content. In the following description, it will be considered that an equipment E may be a single device or a single machine or a set of devices or machines. The term equipment should not be considered in a limiting way. With reference to FIG. 1, the system making it possible to implement the method of the invention notably uses a mobile terminal 1 as defined below.
    [0007] In the remainder of the description, the term "mobile terminal" 1 is understood to mean an electronic device of the mobile phone or touch pad type provided with a screen 11. It may also be glasses, each glass being replaced by a screen. The mobile terminal is provided with a camera and / or video camera for filming the objects located in the real world and displaying the corresponding images on the screen 11 of the mobile terminal. The screen 11 of the mobile terminal 1 will for example be of the touch type. The mobile terminal 1 further comprises processing means, storage means and one or more communication interfaces enabling it to communicate on different wired and / or wireless communication networks. In the remainder of the description, by the term "graphic interface component" W1-Wn (hereinafter generally referred to as W), is meant a "widget" type component that is superimposed in augmented reality on the screen 11 of the mobile terminal 1 to the image of the equipment E displayed on the screen 11. The graphical interface component W for example takes the form of an icon or a context menu to access content linked to the equipment pointed in the real world. This content will for example be stored locally on the mobile terminal 1 or on a remote augmented reality server S1 accessible through a communication network R. In the remainder of the description, reference will be made to a system which preferably comprises a mobile terminal and an augmented reality server S1 communicating with said mobile terminal through a communication network. But it should be understood that the mobile terminal could quite accomplish locally, that is to say offline, the processing and determination of the GUI components to be displayed and the storage in its storage means of the related contents. to each identified GUI component.
    [0008] In the remainder of the description, the term "image" means a frozen image taken using a camera or a video stream comprising several successive images captured with a video camera. The display of a graphical interface component W in augmented reality is realized by launching a specific application on the mobile terminal 1. This specific application is arranged to connect the mobile terminal 1 to the remote augmented reality server S1 used to identify the GUI components W to be displayed on the screen 11 of the mobile terminal 1. The augmented reality server S1 stores an augmented reality database DB in which are stored the graphic interface components W associated with each equipment E or the position and orientation of the mobile terminal 1 in its environment, and the contents associated with each of these components. The identification of the equipment E can be carried out in different ways. The mobile terminal 1 can for example send a request containing the captured image by means of its camera and / or video to the remote server S1 so that it identifies each equipment E represented on the image and returns, in response, each In the other variant embodiment, the mobile terminal 1 sends in the request its geographical coordinates and its orientation and the server S1 determines, from these data, the graphic interface components W to be displayed. . The geographical position and the orientation of the mobile terminal 1 with respect to the targeted equipment can be obtained in various known ways, for example by using chips present in the mobile terminal 1 and / or by using triangulation solutions with the aid of external sensors. Moreover, a matrix code or an electronic tag may be affixed to the equipment E. In the appended figures, we have taken the example of a matrix code but it should be understood that it may be a electronic tag. This matrix code or this electronic tag is read by the user, for example using his mobile terminal 1 by executing a dedicated software application. Generally, this matrix code or electronic tag contains an address that also refers to content stored on the equipment itself or on a remote server S2 to which the mobile terminal 1 connects. In the latter case, the connection is for example made via the wireless communication network R type 3G, 4G, WIFI or any other known solution. The invention will allow the user to access the content related to the matrix code or the electronic tag, even if, for various reasons, the matrix code or the electronic tag is difficult to access. The invention thus consists in transforming the matrix code or the electronic tag into a graphical interface component W available in augmented reality on the screen 11 of the mobile terminal 1 of the user.
    [0009] More specifically, the method of the invention is implemented by executing on the mobile terminal 1 of the user a software application intended to perform this conversion. The software application comprises several software modules executed by the processing means of the mobile terminal 1. The method thus comprises: a step of determining an augmented reality database DB associated with the equipment E. This determination step can to be implemented by recognizing the equipment E located in the real world using the mobile terminal 1 (FIG. 2A) or by determining the geographical position and the orientation of the mobile terminal 1 in its environment. This step is implemented by a determination module responsible for controlling the sending of a request to the augmented reality server S1. As specified above, the request contains data enabling the identification of the equipment E or the position and orientation of the mobile terminal 1 with respect to the equipment. In the first case, it may be for example a photo of the equipment taken by the user using the camera and / or video. A reading step using the mobile terminal matrix code 20 or an electronic tag associated with the equipment and storing the address related to the matrix code or the electronic tag (Figure 2B).
    [0010] This matrix code or this electronic tag is for example affixed to the equipment E and is associated with a content. This content is for example stored on the remote server S2 or on the equipment E itself. In FIG. 2B, the matrix code 20 or the electronic tag respectively codes or stores an address returning to said content stored on the remote server S2 accessible via the communication network R. Reading the matrix code 20 or the electronic tag is realized thanks to a reading module. The reading module controls the camera and / or video of the mobile terminal if it is a matrix code or a suitable reader, for example an NFC reader (Near Field Communication) integrated in the mobile terminal or RFID ("Radio Frequency Identification") type if it is an electronic tag. A storage module stores the address read in the matrix code or the electronic tag. A step of generating a new graphical interface component that can be activated to access said content.
    [0011] In this step, a software application generation module retrieves the stored address and generates a specific graphical interface component Wx for it. This graphical interface component Wx may for example be a symbol or a text, which may include several levels of detail. A step of associating the new widget component Wx equipment E (Figure 2C), a marker affixed in the vicinity of the equipment and recognizable by far or in a geographical reference. A module of the software application allows the user to associate the generated graphical interface component Wx with the equipment E or with a specific recognizable marker. For this purpose, the association module makes it possible, for example, for the user to move the graphical interface component Wx generated on the screen 11 of the mobile terminal 1 to paste it on the fixed or non-frozen image of the equipment E or on a marker displayed on said screen 11, or to position it in a geographical reference. The software application then performs the association and then launches a memory module of this association.
    [0012] Storing the Wx GUI component in connection with said equipment E or with the position and orientation of the mobile terminal 1 in the DB database stored in the augmented reality server 51 (Figure 2D). During this step, the storage module of the software application is responsible for updating the database DB stored in the augmented reality server 51. The new graphical interface component Wx generated, corresponding to the address encoded in the matrix code or stored in the electronic tag, is thus added to the database of the augmented reality server for the selected equipment E.
    [0013] The matrix code may for example be a QR code ("Quick Response Code") or another type of known two-dimensional code. The electronic tag will for example be a NFC type tag ("Near Field Communication") or type RFID ("Radio Frequency Identification").
    [0014] The invention thus makes it easy to access the content related to the matrix code 20 or to an electronic tag present on a device, without having to access it directly. The user can thus view the content related to the matrix code or the electronic tag by selecting the graphical interface component Wx appearing in augmented reality on the screen 11 of his mobile terminal 1 when the latter points the equipment. E.
    权利要求:
    Claims (12)
    [0001]
    REVENDICATIONS1. A method for generating augmented reality content for a real-world equipment (E) by employing a mobile terminal (1) having a screen (11), characterized in that it comprises: a step of determining an augmented reality database stored in an augmented reality server (S1), a step of reading using the mobile terminal of a matrix code (20) or an electronic tag associated with said equipment (E) , said matrix code (20) or said electronic tag respectively coding or comprising an address linked to a content, a step of storing said address, a step of generating a graphical interface component (Wx) corresponding to said address allowing accessing said content, a step of storing the graphical interface component (Wx) in the augmented reality database (DB) stored in the augmented reality server (S1).
    [0002]
    2. Method according to claim 1, characterized in that the matrix code (20) is a QR code or equivalent.
    [0003]
    3. Method according to claim 1, characterized in that the electronic tag is of the NFC or RFID type.
    [0004]
    4. Method according to one of claims 1 to 3, characterized in that it comprises a step of association of the graphical interface component (Wx) with the equipment (E) or with a marker affixed in the vicinity of the 'equipment.
    [0005]
    5. Method according to one of claims 1 to 4, characterized in that the step of associating the graphical interface component (Wx) with the equipment (E) is performed by moving the graphical interface component ( Wx) displayed in augmented reality on the screen (11) of the mobile terminal (1) on an image of the equipment (E) displayed on the screen (11) of the mobile terminal (1) or in a geographical reference.
    [0006]
    6. Method according to one of claims 1 to 5, characterized in that the graphical interface component (Wx) is composed of a text or a symbol.
    [0007]
    7. augmented reality content generation system for a device located in the real world, characterized in that it comprises: a module for determining an augmented reality database (DB) stored in a server (S1); ) of augmented reality, - a module for reading a matrix code (20) or an electronic tag associated with said equipment (E), said matrix code (20) or said electronic tag respectively coding or comprising an address linked to a content, said electronic tag being associated with a content, - a module for storing said address, - A module for generating a graphical interface component (Wx) corresponding to said address for accessing said content, - A module storing the GUI component in the augmented reality database (DB) stored in the augmented reality server (S1).
    [0008]
    8. System according to claim 7, characterized in that the matrix code (20) is a QR code or equivalent.
    [0009]
    9. System according to claim 7, characterized in that the electronic tag is of the NFC or RFID type.
    [0010]
    10. System according to one of claims 7 to 9, characterized in that it comprises an association module of the GUI component (Wx) with the equipment (E).
    [0011]
    11. System according to one of claims 7 to 10, characterized in that the association module of the GUI component (Wx) with the equipment (E) is arranged to allow a sliding of the GUI component. (Wx) displayed in augmented reality on the screen (11) of the mobile terminal (1) on an image of the equipment (E) displayed on the screen of the mobile terminal.
    [0012]
    12. System according to one of claims 7 to 11, characterized in that the graphical interface component (Wx) is composed of a text or a symbol.
    类似技术:
    公开号 | 公开日 | 专利标题
    EP2924626B1|2018-10-03|Method for generating a content in augmented reality mode
    US10007664B2|2018-06-26|Systems, methods and apparatus for dynamic content management and delivery
    US20150009152A1|2015-01-08|Method of sharing application and electronic device using the same
    US20110170787A1|2011-07-14|Using a display to select a target object for communication
    US20140270477A1|2014-09-18|Systems and methods for displaying a three-dimensional model from a photogrammetric scan
    US9870549B2|2018-01-16|Virtual postage based on image recognition
    US9236911B2|2016-01-12|Secure transfer of whiteboard content
    US8185596B2|2012-05-22|Location-based communication method and system
    US20190012756A1|2019-01-10|Method and system for providing information regarding with delivery article using augmented reality
    CN105827959B|2019-06-25|Method for processing video frequency based on geographical location
    US20140267796A1|2014-09-18|Application information processing method and apparatus of mobile terminal
    US20140089096A1|2014-03-27|Method and apparatus for using identifiers associated with products for applying a mobile device skin
    US9129178B2|2015-09-08|Method and apparatus for processing data using optical character reader
    EP2818965B1|2018-10-31|Method for interaction between a digital object, representative of at least one real or virtual object located in a remote geographical perimeter, and a local pointing device
    WO2016101735A1|2016-06-30|Drawing data based instant messaging method, client and system
    FR3074332B1|2019-11-15|METHOD FOR POSITIONING AT LEAST ONE VIRTUAL OBJECT IN AT LEAST ONE VIRTUAL AREA FOR RECEIVING VIRTUAL OBJECTS, TERMINAL AND ASSOCIATED SYSTEM
    KR102133718B1|2020-07-21|Apparatus and method for providing information using barcode
    US20180189321A1|2018-07-05|Data models for geo-enriched data
    US10659409B1|2020-05-19|Media access system
    CN107767162A|2018-03-06|Advertisement acquisition method and device based on packing box
    CN107169691A|2017-09-15|system and method for information processing
    CN107911818A|2018-04-13|The method, device and mobile terminal of data transfer between mobile terminal
    EP2817951B1|2017-01-04|Telecommunication system
    US20150325049A1|2015-11-12|Augmenting a digital image
    FR2987924A1|2013-09-13|Human-machine interface generating method for use in mobile terminal e.g. tablet, involves displaying user interface component, and locating display of graphical user interface on periphery of detected position of finger
    同族专利:
    公开号 | 公开日
    CN104951498A|2015-09-30|
    CN104951498B|2019-04-05|
    FR3019333B1|2016-03-11|
    US20150279112A1|2015-10-01|
    US9779551B2|2017-10-03|
    EP2924626B1|2018-10-03|
    EP2924626A1|2015-09-30|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20120026530A1|2010-07-27|2012-02-02|Xerox Corporation|Augmented reality system and method for device management and service|
    US20120181330A1|2011-01-14|2012-07-19|John S.M. Chang|Systems and methods for an augmented experience of products and marketing materials using barcodes|
    FR2987924A1|2012-03-08|2013-09-13|Schneider Electric Ind Sas|Human-machine interface generating method for use in mobile terminal e.g. tablet, involves displaying user interface component, and locating display of graphical user interface on periphery of detected position of finger|
    US7236854B2|2004-01-05|2007-06-26|Abb Research Ltd.|Method and a system for programming an industrial robot|
    KR101329882B1|2010-08-12|2013-11-15|주식회사 팬택|Apparatus and Method for Displaying Augmented Reality Window|
    US8493353B2|2011-04-13|2013-07-23|Longsand Limited|Methods and systems for generating and joining shared experience|
    US9827487B2|2012-05-14|2017-11-28|Sphero, Inc.|Interactive augmented reality using a self-propelled device|
    CN102739872A|2012-07-13|2012-10-17|苏州梦想人软件科技有限公司|Mobile terminal, and augmented reality method used for mobile terminal|
    US9317972B2|2012-12-18|2016-04-19|Qualcomm Incorporated|User interface for augmented reality enabled devices|JP2018005091A|2016-07-06|2018-01-11|富士通株式会社|Display control program, display control method and display controller|
    CN108021226B|2016-11-03|2021-07-20|深圳劲嘉盒知科技有限公司|AR content interaction realization method and device based on product packaging|
    CN106502413A|2016-11-07|2017-03-15|浙江网仓科技有限公司|Wearable warehouse operation method and system|
    US11062517B2|2017-09-27|2021-07-13|Fisher-Rosemount Systems, Inc.|Virtual access to a limited-access object|
    US11244509B2|2018-08-20|2022-02-08|Fisher-Rosemount Systems, Inc.|Drift correction for industrial augmented reality applications|
    法律状态:
    2015-02-09| PLFP| Fee payment|Year of fee payment: 2 |
    2016-02-10| PLFP| Fee payment|Year of fee payment: 3 |
    2017-02-07| PLFP| Fee payment|Year of fee payment: 4 |
    2018-02-06| PLFP| Fee payment|Year of fee payment: 5 |
    2019-02-07| PLFP| Fee payment|Year of fee payment: 6 |
    2020-12-18| ST| Notification of lapse|Effective date: 20201110 |
    优先权:
    申请号 | 申请日 | 专利标题
    FR1452555A|FR3019333B1|2014-03-26|2014-03-26|METHOD FOR GENERATING INCREASED REALITY CONTENT|FR1452555A| FR3019333B1|2014-03-26|2014-03-26|METHOD FOR GENERATING INCREASED REALITY CONTENT|
    EP15157176.7A| EP2924626B1|2014-03-26|2015-03-02|Method for generating a content in augmented reality mode|
    US14/665,143| US9779551B2|2014-03-26|2015-03-23|Method for generating a content in augmented reality mode|
    CN201510134964.XA| CN104951498B|2014-03-26|2015-03-26|Method for generating the content under augmented reality mode|
    [返回顶部]